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ABSTRACT 

This bulletin focuses on three developments related 
to evaluation. First, the plans and initial work of the Mathematical 
Sciences Education Board, formed in 1985, are described. The Board's 
goals, its view of the major challenges in mathematics education 
lacing the nation's schools, and its plans are discussed. The 
development of a design ff^r studying the impact of testing on 
mathematics education is then described. The second section of the 
bulletin provides an overview of a revised publication on evaluation 
in the mathematics classroom, available from ERIC/SMEAC ("Evaluation 
in the Mathematics Classroom," published fivst in 1974). What is 
included in the publication is noted, with several illustrative 
examples. A second publication with similar goals ("Mathematics 
Assessment for the Classroom Teacher," developed by the Virginia 
Council of Teachers of Mathematics) is also described. In the third 
section of the bulletin, the need to evaluate mathematical problem 
solving more effectively is considered. Procedures that can be used 
in addition to paper-and'pencil maasures are listed. Finally, several 
tests that Attempt to assess mathematical problem solving processes 
are cited, with examples. Sixteen references are listed. (MNS) 
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Evaluation In Mathematics Instruction 



In 1985, an overview of data from 
state assessments in mathematics was 
deveioped (Suydam, 1984). Such as- 
sessments have focused attention on 
evaluation across classrooms, as have 
the three sets of reports from the Na- 
tional Assessment of Education Pro- 
gress (NAEP). Data from the fourth na- 
tional mathematics assessment will ap- 
pear in the near future, and these data 
will surely be reviewed as carefully as 
the previous assessments have been. 
Information both on status - how well 
are students achieving currently on the 
various mathematical ideas they need 
to know - and on change - what, if any, 
progress has been made since previous 
assessments - is vital information that 
has been and will be used to improve 
instruction. 

Moreover, data from the Second In- 
ternational Study of Mathematics 
(SIMS) attained headlines, as did data 
from the First International Study, when 
the ranking of the United States placed 
it well below almost every other country 
on most of the achievement scales. 
Concern for what is going on in class- 
rooms has led to many studies in the 
past decade, and evaluation is one 
facet that has been investigated. 

Aside from state, national, and inter- 
national assessments, evaluation goes 
on, day afti&r day. In each of our class- 
rooms. Such evaluation is addressed by 
the revision of E¥aluatlon In the 
Mathematics Claasroom: From What 
and Why to How and Where (Suydam, 
1636). it attempts to provide useful 
guidelines and techniques for class- 
room teachers. 

This bulletin will provide an overview 
of the types of information available 
from that publication. Then recent work 
oh one aspect of evaluation will be con- 
eldoi^d - how problem solving can be 
evaluated. But first, consideration will 
be given to the plana and Initial work of 
a; new ajgency In mathcrniatlca educa- 
tibri, with particular note of Its planned 
w6rt( pertaining to evaluation and 
tetitlriOi In particular 



IMathematicai Sciences Education 
Board 

The Mathematical Sciences Educa* 
tion Board (MSEB) was established in 
October 1985, at the urging of the 
mathematics community, as a major 
unit of the National Research Council 
(which In turn is the principal operating 
agency of the National Academy of 
Sciences and the National Academy of 
Engineering). The Board is designed to 
grow into the most comprehensive and 
far-reaching national leadership effort 
in mathematics education ever to be 
undertaken in the United States. Much 
of the material which follows is drawn 
from the informational materials dis- 
tributed by the MSEB. 

Through a variety of efforts, the 
Mathematical Science Education Board 
is seeking to provide: 

• leadership of continuing efforts to 
improve mathematical sciences edu- 
cation nationally; 

• coordination among on-going educa- 
tional projects; 

• service to localities and states, 
through assistance In determining 
core competencies for all students 
and higher standards, and enhancing 
teacher preparedness; 

• recommendations on how to 
strengthen weak parts of the infra- 
structure of mathematics education; 

• Intormatlon to enhance public under- 
standing of the rapidly changing 
character of the mathematical 
sciences; 

• advice to federal, state, and local 
agencies on long-range goals and 
needs in mathematical science 
education. (MSEB, 1986a, p. 2) 
The intent is to provide "for sus- 
tained attention over many years to the 
major issues affecting tha quality of in- 
struction and learning In the mathe- 
matical sciences" (MSEB, 1986a, p. 2). 
National In scope, the MSEB is action- 
oriented, concerned not only with mak- 
ing recommendations but also with im- 
plementing Ideas. 

The MSEB views the nation's schools 
as facing mflijor challenges in temns of: 

PERMISSION TO REPRODUCE THIS ; ; V . : 



Curricula and instruction in our 
schools are years behind the times. 
They have not been modified and up- 
dated to reflect the increased de- 
mand for employees and citizens 
who possess well-developed prob- 
lem-solving and higher order thinking 
skills in mathematics. Nor do they 
reflect the greatly expanded uses of 
the mathematical sciences. The 
skills that are needed go well beyond 
the sort of facility with calculation 
and algebraic manipulation most 
people identify as 'mathematics.' 
Calculators and computers have had 
virtually no impact on mathematics 
classrooms, in spite of their potential 
to greatly enrich, enlighten, and 
enlarge school mathematics. Further- 
more, there is little prospect that 
they will, unless there is continued 
strong national leadership to do two 
things: 

- Focus attention on the remarkable 
opportunities computers offer to 
displace some tedious drill; to 
teach more sophisticated con- 
cepts; to provide individualized in- 
struction; to use mathematics to 
solve a plethora of problems here- 
tofore unsolvable by students; and 
to put more of a sense of wonder 
and discovery into the learning of 
mathematics. 

-Chart a course for the develop- 
ment of new curricula, instruc- 
tional techniques, and tools for 
evaluation which will be suffi- 
ciently widely accepted so that 
textbook publishers and software 
companies, etc., will be willing to 
invest the necessary time and 
resources. 
• Methods of evaluation - especially 
standardized, paper-and-pencil, 
multiple-choice tests of 'basic skills* 

- are being used across the country 
without sufficient reflection and are 
themselves obstacles to the teaching 
of new methods and higher order 
thinking skills, as well as to the use 
of calculators and computers. The 
nation is in the grip of a ieating 
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mystique' Which has led to the wlde^ 
spread use of such tests In spite of 
repeated warnings that several 
premises upon which their use Is 
based are open to serious question. 
• Finally, and most Important, there Is 
little awareness In thd schools - and 
virtually none In the general public - 
of what lies Just ahead: mathematics 
curricula, Instruction, and methods 
of evaluation must undergo dramatic 
change If we are to properly educate 
tomorrow's employee and tomor- 
row's citizen. (MSEB, 1986a, pp. 2-3) 



The Board is planning to move 
"through a succession of planned 
stages of change until learning levels In 
school mathematics have been brought 
much more Into line with the reality of 
national needs and national potential" 
(MSEB, 18S6a, p. 6). Working coopera- 
tively with the National Council of 
Teachers of Mathematics (NCTM), the 
Board plans to develop and publish, by 
the end of 1988, recommended 
standards or criteria for excellence In 
school mathematics, K-12. These stand- 
ards will cover curriculum, instruction, 
and evaluation. One part of this 
massive effort will involve the develop- 
ment of ''guidelines vjt redesigning 
tests and other assessment mecha- 
nisms so that they are properly aligned 
with th9 curriculum and provide mean- 
ingful evaluation of student achieve- 
ment" (MSEB, igsea, p. 7). 

The MSEB is currently developing 
the design for studying the Impact of 
testing on mathematics education. The 
purpose of the proposed set of studies 
Is to: 

document Information about 
current tests and testing prac- 
tices, assess the impact of 
these practices on various 
groups, and direct future 
developments so that testing 
practices will be less disruptive, 
more helpful, and more effec- 
tive. (MSEB, 1906b, p. 1) 
The report notes that the specific ac- 
tivities planned have emerged from the 
following perspectives: 

• Valid information about student 
achievement in mathematics Is 
needed by a variety of people (stu- 
dents, teachers, parents, adminis- 
trators, policy makers) for a variety of 
purposes (monitoring progress, se* 
lection for placement in courses, pro- 
gram evaluation, accountability). 

• Both the curriculum and teaching 
practice In mathematics need to be 
directed toward the learning of 
strateglea to oolve problems, the ap* 
plication of mathematica to practical 
situations, and the development of 
thinking akilla. Consequemiy, tests 
should reflect atudenta' achievement 
In thoioib diractlone. 



• Serious questions have arisen about 
the validity of existing tests for the 
uses to which they are being put. 
Standardized tests and state- 
mandated tests may yield Informa- 
tion that Is Invalid for certain pur- 
poses, and they provide little or no 
Information on several Important dl-^ 
mansions of achievement. 

• Furthermore, the continued use of 
existing tests appears likely to Im- 
pede the much-needed reforms In 
curriculum and Instruction that the 
mathematics education community 
Is developing. (MSEB, 1986b, pp. 1-2) 
Additional Information about the on- 
going work of the Mathematical 
Sciences Education Board, and detailed 
plans for Its various projects may be ob- 
tained from: 

Mathematical Sciences Educatfon board 
National Research Courrjji 
2101 Constitution Avenue, N.W. 
Washington, D.C. 20418 
(202) 334-3294 
Some readers of this bulletin may be 
wondering why so much attention was 
directed toward the MSEB. Clearly, It 
promises to exert a force, operating 
from the national level, that we have not 
had In mathematics education. And If 
the planned efforts succeed, then 
evaluation processes, and testing In 
particular, will change. All of us need to 
keep aware of the work of the MSEB 
and provide both advice and aid from 
our varying perspectives In schools at 
one level or another. The process of 



EDITOR'S NOTES 

ERIC users Interested In mathe- 
matics education may be Inter- 
ested In knowing that. In addition 
to the focus on testing and evalu- 
ation described In this Informa- 
tion bulletin, the Mathematical 
Sciences Education Board Is also 
Initiating work In four additional 
strands: national needs and po- 
tential, currlcular frameworks and 
Instruction, the teaching profes- 
sion, and colleglrite mathematics. 
Projects will vary both In duration 
and type, from one-year research 
programs to three-year compre- 
hensive studies, series of topical 
conferences, and one-shot con- 
sciousness-raising symposia. The 
Mathematical Sciences Education 
Board Is committed to the Idea 
that the changes which must take 
place In mathematics education 
over the next 20-30 years can be 
accomplished smoothly only by 
means of a coordinated effort In- 
volving all of the principal actors: 
state and !ocal educational units, 
professional societies, colleges 
and universities, parent groups, 
publishers, testmakers, etc. 
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education can benefit from collective 
collaboration and strong, responsive 
leadership. 

Overview of Evaluation In the 
Mathematics Classroom 

Izvalusiion In the Mathematics Class- 
room was first published In 1974: after 
12 years. It was clearly time to revise It. 

In the Introductory section, evalua- 
tion Is Illustrated by some classroom 
vignettes, and the purposes of the 
booklet are presented: 

(1) to help readers develop better 
paper-and-pencll measures for 
evaluation, and 

(2) to review with readers other ap- 
proaches to evaluation. 

The Idea that evaluation Is much more 
than paper-and-pencll tests is 
emphasized. 

The second section continues with 
this idea by considering the scope of 
evaluation. Subsections deal with the 
scope of mathematics objectives to be 
evaluated, the scope of evaluation pur- 
poses, and the scope of evaluation pro- 
cedures. Observations, Interviews, in- 
ventories and checklists, attitude 
scales, criterion-referenced tests, norm- 
referenced tests, standardized tests, 
and diagnostic tests are among the 
types of evaluation procedures 
discussed. 

The third section of the publication 
concerns developing tests. How to plan 
a test Is discussed first, with suggest 
tions on listing the objectives to be 
assessed, the types of Items to be con- 
structed, and deciding on the number 
of Items to be written for each objec- 
tive. Twenty-one general suggestions 
for writing test Items are then 
presented. Such points as the following 
are made: 

• Use clear, simple statements. Use 
language that students understand. 
C hoose CO nc I se vocabu lary, and 
sentence construction that Is appro- 
priate to the level of your students. 
Break a complex sentence Into two 
or more separate sentences. 

• Design each Item so that It provides 
evidence that an objective has been 
achieved. Avoid testing for unimpor- 
tant details, unrelated bits of infor- 
mation, or Irrelevant material. 

• Begin a test with easy Items. Placing 
difficult Items at the beginning of a 
test Is likely to discourage average 
and below-average achievers. You 
can then arrange items so that the 
test gets Increasingly more difficult, 
or you can mix easy and difficult 
Items. 

• Analyze student responses to each 
item, for diagnostic use. 

If such suggestions seem familiar. It 
may be because they have become 
familiar In a variety of publications on 
deveioping good tests, such as 
Qronlund (1984). 



Following these general points are 
specific suggestions for developing 
each type of test that might be used in 
mathematics classrooms: 

• short-answer questions or comple- 
tion Items 

• multlple-choice items 

• true-false Items 

• matching items 

• essay items 

Finally, some related points about 
testing are considered: item pools, item 
analysis as it can be done with teacher- 
made tests, and test bias. Some illus- 
trative examples are Included through- 
out. 

The brief concluding comment notes 
that the goal of evaluation Is that of im- 
proving instruction. Measuring or 
assessing or testing only Indicates: the 
teacher then has to do something as a 
result of what has been indicated. 
Evaluation Is one stage In the on-going 
process of teaching. 

Lists Of references, with annotations, 
are provided In the final section of the 
publication. While a few of these 
references provide general background 
information, most were selected 
because they Were written to help 
classroom teachers do a better job of 
evaluating and making use of that 
evaluation. 

Another publication with some 
similar Intentions and information was 
developed by the Virginia Council of 
Teachers of Mathematics (VCTM, 1983). 
Titled MQthemfitlcs Assessment for tha 
CiGSsroom Teacher, It was written by a 
number of persons, each contributing 
one of the following chapters: 

• Legal Ramifications of Evaluation 
(William Hefty . a city attorney U) 
Richmond, Virgina) 

• Construction and Interpretation of 
Teacher-Made Tests (James Impara) 

• Classroom Assessment of Concep- 
tual Understanding (John VandeWalle 
and Harold Mick) 

• Diagnosis in the Mathematics Class- 
room (John McGregor) 

• Affective Evaluation (Stuart Flanagan) 

• Assessing Problem Solving Skills 
(Ena Gross) 

• Grading and Reporting (Josephine 
Baker) 

• Mathematics Achievement: Assess- 
ment Strategies and Applications 
(Anne Polselli Sweet) 

Among the noteworthy features of this 
publication is that It Is filled with 
specific examples of classroom prac- 
tices and with specific items that Il- 
lustrate various Points. In the preface Is 
the note that It was developed In order 
to help teachers implement one of the 
recommendatioha made by the Na- 
tional Council Of Teachers of Mathe- 
matics in An Agenida tor Action: 



Recommendations for School Mathe- 
matics of the 19B0S so that 
the success of mathematics pro- 
grams and student learning be 
evaluated by a wider range of 
measures than conventional 
testing. (NCTM, 1980) 

The Council underlines this recommen- 
dation with the following comments: 
Evaluation is not limited to testing. 
It Includes gathering data and in- 
terpreting the data. Testing Is one 
source of data. There are many 
others. Today, many people use 
test scores as the sole Index of the 
quality of mathematics programs 
or of the success of student 
achievement. Test scores alone 
should not be considered 
synonymous with achievement or 
program quality, (p. 13) 

It Is imperative that the goals of 
the mathematics program dictate 
the nature of the evaluations 
needed to assess program effec- 
tiveness, student learning, teacher 
performance, or the quality of 
materials. Too often the reverse Is 
true: the tests dictate the program, 
or assumptions of the evaluation, 
plan are Inconsistent with the pro- 
gram's goals, (p. 14) 
When one considers that the first of the 
Councirs recommendations Is that 
"problem solving be the focus of 
school mathematics In the 1980s," 
there Is realization that tests often fall 
to assess this focus effectively. 

Evaluating Problem Solving 
In An Agenda for Action, cited above. 

It Is noted that 
an emphasis on problem solving 
demands more flexibility and 
creativity In assessment than Is 
possible within the restrictions of 
most current test formats. (NCTM, 
1980, p. IS) 

Moreover, It Is recommended that 
Longitudinal evaluation of In- 
dividual problem-solving ability 
should be developed. The acquisi- 
tion of problem-solving skills Is a 
long-term process and should not 
be evaluated solely with short-term 
measures, (p. IS) 

Such comments promoted aware- 
ness of the limitations of many prob- 
lem-solving programs, as well as of the 
evaluation of problem solving. Ttne 
Mathematical Sciences Education 
Board has reaffirmed that focusing ih^ 
curriculum on problem solving means 
that we must do something different so 
that problem solving is appropriately 
assessed. 

Your goals need to be considered as 
you plan how to evaluate. If you want 
children to solve both routine and 
nonroutine problems, then you must In- 
clude both types of problems In your 



evaluation. If you are emphasizing the 
teaching of problem-solving strategies, 
then you must plan to assess students' 
ability to use strategies. 

In addition to paper-and-pencil 
maasures, such procedures as the 
following should be considered: 

• Face students with a problem-solving 
situation and observe how they meet 
It. Such points as these might be 
considered: 

- Is there evidence of careful reading 
of the problem? 

-Do individual children seem to 

have some means of beginning to 

attack a problem? 
-Do they apply a strategy, or do 

they try to use the last procedure 

they were taught? 

- Do they have another strategy to 
try If the first one falls? 

- How consistent and persistent are 
they In applying a strategy? 

^Are careless errors being made, 
and If so, when and why? 

-How long are they willing to keep 
trying to solve a problem? 

-How well are they concentrating 
on the task? 

- How quickly do tiiey ask for help? 
-What strategies does each child 

use most frequently? 

- Do they use manipulative materials? 
-What do their behaviors and such 

factors as the expressions on their 
faces Indicate about their Interest 
anu Involvement? (Reys et al., 
1984, p. 33) 

• Interview students. Interviews let you 
delve Into how a student goes about 
solving a problem, and why he or she 
does It that way. 

-Basically, you need to (1) present 
the student with a problem; (2) let 
the student find a solution, 
descricing what he or she Is doing; 
and (3) challenge the student, 
ellcltlhg specific details on what 
ho or she Is doing and why. (Reys 
et al., 1984, p. 33) 
It l3 helpful to have an exact record 
of what Is said, so notes should be 
taken or a tape-recorder used. This 
makes It easier to analyze the 
student's thinking later, perhaps 
morfi carefully and from a different 
perspective than when Involved In 
the imervlew. 
«2 Have students describe to a group 
how they solved a problem. Again, 
taking notes will be helpful. 
>^ Have one student teach another how 
to solve a problem. Observing this 
process not only can provide Infor- 
mation on the students' thinking, but 
also on the teaching strategies 
they've absorbed! 

• Make an Inventory or checklist. An In- 
ventory can be used to che^k on 
whether a student can solve a prob> 
lem with a specified strategy, or 
solve a problem using two or three 



stratagles. The goal is merely the 
answer the emphasis Is on finding 
out whether or not students can use 
the strategies. 

Paper-and-pencll tests can also be 
used to assess the ability to solve prob- 
lems. In recent years, several tests have 
been developed that attempt to assess 
various aspects of Polya's four-step 
model of problem solving: 

• First, understand the problem. 

• Second, devise a plan for solving it. 

• Third, carry out your plan. 

• Fourth, look back to examine the 
solution obtained. (Polya, 1973) 

One test of this type which is widely 
used was developed by Wearne and 
Romberg (Weame, 1977). It provides in- 
formation about the child's mastery of 
the prerequisites of the problem-solving 
questions posed. To provide this addi- 
tional information, each problem- 
solving question is preceded by two 
other questions. One question 
assesses the child's understanding of 
the information contained in the item 
stem of the problem-solving question; a 
second question assesses the child's 
knowledge of an underlying concept of 
the problem-solving question. 

Schoen and Oehmke (1980) reported 
on the development of the Iowa Prob- 
lem Solving Test. The goal was to 
"produce an easily administered test 
that provides information about the 
problem-solving subskills that is highly 
correlated with data from individual in- 
terview settings" (p. 218). Three scores 
are provided for each student, dealing 
with understanding problems, applying 
the solution strategies chosen, and 
looking back at the solution. 

Sample Items are Included for each 
subtest; for Instance, this is the sample 
Item for "looking back", which requires 
the student to identify problems that 
can be solved In the same way as a 
given problem, to determine the effect 
of varying the conditions in a given 
problem, or to evaluate a given solution 
strategy. 



Shelley has 75 marbles, which is 11 
more than twice as many as Karen 
has. To find how many marbles Karen 
has, Shelley added 75-^11 and got 
86. She then said Karen has 43 
marbles. Is Shelley right? 

1) Yes. 

2) No. She should have multiplied 
86x2 and got 172. 

3) No. She should have subtracted 
75-11 =64. Then 32 Is the right 
answer. 

4) No. She should have multiplied 
1lx2s22. Then 75- 22 = 53 is 
the right answer. 



A somewhat similar test has recently 
been developed by Hofmann (1966). In- 
stead of developing three subtests 
which cluster items of a particular type, 
she clustered items assessing fouf 
stages of the problem-solving process - 
read, select a strategy, solve, and 
review and extend - with each problem. 
An item from her pool of 200 items 
being verified is: 

A pirate found a treasure chest con- 
taining silver coins. Ke buried half of 
them and gave half of the remaining 
coins fo his mother. If he was left 
with 4500 coins, which he put in a 
black sack, how many were in the 
treasure chest that he found? 

1) To solve this problem, you need to 
know that the: 

a) treasure chest had no gold 
coins. 

b) black sack had half of the 
original coins. 

c) pirate buried more coins than 
he gave his mother 

d) number of coins the pirate 
found was less than 5000. 

2) To solve this problem, you would 
most likely: 

a) start with the number of coins 
in the pirate's black sack and 
then work backwards. 

b) draw a picture of a treasure 
chest. 

c) pretend the pirate had fewer 
coins. 

d) guess a number and then 
check to see if it is correct. 

3) The number of coins in the trea- 
sure chest when the pirate found 
it was: 

a) less than 5000. 

b) more than 5000 but less thr^n 
9000. 

c) exactly 9000. 

d) more than 9000. 

4) After solving the problem, you 
know that the pirate gave his 
mother 

a) 1/2 of the coins in the chest. 

b) 1/3 of the coins in the chest. 

c) 1/4 of the coins in the chest. 

d) 1/5 of the coins in the chest. 

Perhaps it Is apparent that the type 
of problems used by Schoen and 
Oehmke and by Hofmann differ. The 
problem with the marbles is somewhat 
similar to Items found In many text- 
books, while the problem of the pirate 
and the coins is nonroutine. The type of 
problem must be considered when 
selecting a test, for the test must 
assess what has been taught. Results 
of Hofmann's test would probably not 
be comparable for students who had 
had experience with nonroutine 
problems and those who lacked such 
experience. 

The need to match test and curric- 
ulum has been emphasized by several 



analyses. Knifong (1980), for instance, 
found considerable variation in compu- 
tational procedures and in difficulty 
level among the word problem sections 
of eight standard achievement tests for 
grade 6. Some problem-solving objec- 
lives on the North Carolina Minimura 
Competency Test had little coverage in 
the two textbook series which were 
analyzed (and widely used in North 
Carolina schools), according to Rogers 
(1981). And in a detailed analysis of the 
content of four textbooks and five tosts 
for grade 4, Freeman et al. (1983) 
reported that: 

• 6 topics were in all textbooks and 
tests; 

• 3 topics were in books but not tests; 

• 3 topics were on tests, with limited 
attention in books; 

• 10 topics were in a!! books, some 
tests; and 

• 385 topics were 'covered' by a single 
textbook or a single test! 

Thus, it is clearly both important and 
necessary to assure that what is tested 
is being taught. 

One further comment about the 
development of multiple-choice tests 
should be made. In addition to develop- 
ing tests which assess the problem- 
solving steps or the use of strategies, it 
is also possible to develop tests to 
assess particular understanding about 
ihe mathematical concepts involved in 
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the problem (acknowledging that some 
would term this ''understanding''). One 
such item Is: 

100 is multiplied by a number smaller 
than 1. 

The answer has to be: 

a) grt»ater than 100 

b) less than 100 

c) a fraction 

d) 0 

Items can also be developed to assess 
such problem-solving skills as: 

• Identifying needed Information from 
extraneous Information 

• estimating answers 

• recognizing a reasonable answer 

• making valid conclusions from the in* 
formation given 

• ... and so on through the list of skills 
students should know. 

Once again, the emphasis is on 
matching what is evaluated with what 
needs to be evaluated. 

Concluding Comment 

In several different ways, this bulletin 
has attempted to promote awareness 
that there are many methods that 
should be used in iTiathematics class- 
rooms ^o evaluate the effectiveness of 
instruction. If we are to change percep- 
tions about mathematical this may be 
imperative. The Mathematical Sciences 
Education Board believes that it is. The 
approach the Board is taking of con- 
sidering curriculum, instruction, and 
evaluation may be the key to changing 
all three. Both assessment data and 
observations in classrooms Indicate the 
necessity of this. 
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